App. No. 09/667,424 

Amend, dated Apr. 13, 2005 

Resp. to Office Action dated Nov. 18, 2004 

REMARKS/ARGUMENTS 

Reconsideration is respectfully requested of the Office Action of November 18, 2004. 
A request for a two month extension of time, together with the associated fee, is filed 
herewith. 

Applicants acknowledge with appreciation the courtesy extended to counsel by Examiner 
Selby and the Primary Examiner Mr. Tuan Ho at the interview conducted on March 22, 2005. 
Claim 1 and the Miura reference, U.S. patent 4,879,596, were discussed at the interview at the 
conclusion of which the Examiners agreed that the proposed amendments to Claim 1, adding the 
limitation of an image processing means for calculating a three-dimensional distance distribution 
and the limitation of the corresponding area being searched in a striplike search area was not 
disclosed by Miura, 

The claims in the case are: 1, 3, 5-7, 9, 10, 13, 16, 21 and 22. Claims 1 and 6 have been 
amended to further clarify the features of the present invention. New Claims 21 and 22 are 
presented for the purpose of claiming further detailed aspects of the invention not rendered 
obvious by the prior art. 

The claim amendments are fully supported by the original claims and the specification. 
No new matter has been added. The Examiner is respectfully requested to reconsider and 
withdraw the outstanding objections and rejections in view of the remarks contained herein. 
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Double Patenting (Provisional Rejection) 

It is noted that the rejection of Claim 5 under the judicially created doctrine of 
obviousness-type double patenting has not been repeated and is therefore considered to be 
withdrawn. 

No Issues Under 35 USC § 112. 

It is noted that none of the claims have been objected to under 35 U.S.C. § 112. 

Claim Rejections - 35 U.S.C. § 103 

The following rejections were set forth in the Office Action of November 1 8, 2004: 

1) Claims 1, 3, 5, 9 and 16 were rejected under 35 U.S.C. § 103(a) as allegedly 
unpatentable over Miura (US 4,879,596). 

2) Claims 2, 4, 10-13 and 14 were rejected under 35 U.S.C. § 103(a) as allegedly 
unpatentable over Miura (previously cited), in view of Tanigawa (US 5,915,033). This rejection 
is moot with respect to Claims 2, 4, 11, 12 and 14, as these claims have been cancelled. 

3) Claim 6 was rejected under 35 U.S.C. § 103(a) as allegedly unpatentable over Mzwra, 
in view of Lipton (US 6,063,441). 

4) Claims 7, 15 and 17-20 were rejected under 35 U.S.C. § 103(a) as allegedly 
unpatentable over Mmra, in view of Saneyoshi (US 5,410,346). 

All these rejections are traversed and reconsideration is respectfully requested in view of 
the foregoing amendments. 

The rejections of Claims 2, 4, 11, 12, 14, 15 and 17 to 20 are rendered moot by 
cancellation of these claims. 
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Resp. to Office Action dated Nov. 18, 2004 

By this response, Claim 1 has been amended as follows: 
1. (Currently Amended) A stereo camera apparatus comprising: 
a main camera for taking a photograph of an object; and 

a sub-camera for taking a photograph of said object from a point of view different from a 
point of view of said main camera, said main camera and said sub-camera being disposed with 
respect to each other by a predetermined spacing, 

a shooting direction of said stereo camera is substantially perpendicular to said 
predetermined spacing in a baseline between the main camera and the sub-camera, 

image processing means for calculating a three-dimensional distance distribution of said 
object based on a positional difference between a region in a reference image photographed by 
said main camera and a corresponding area in a comparative image photographed by said sub- 
camera to an image signal of said region. 

wherein said corresponding area is searched in a striplike search area having a 
predetermined length which extends from a position substantially corresponding to said region, 
said positional difference is obtained from an area which is capable of setting said search area 
inside of said comparative image, 

wherein optical axes of said main camera and said sub-camera are inclined toward the 
main camera side with a predetermined angle with respect to the shooting direction defined by 
each of the optical axes and the shooting direction^ 

wherein angles of inclination of said main camera and said sub-camera are set to be such 
angles that said three-dimensional distance distribution is substantially left-right symmetric with 
respect to the shooting direction. 
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Resp. to Office Action dated Nov. 18, 2004 

Applicants submit that Claim 1 is not suggested, taught or hinted at in Miura and 
accordingly Miura fails to estabUsh prima facie obviousness of the claimed invention. 

The camera structure of Miura provides for one camera that can be rotatably adjusted in 
an up-down direction (vertical direction), and the other camera can be rotatably adjusted in a 
left-right direction (horizontal direction), and both cameras can be rotatably adjusted with an 
optical axis of each camera as a center of rotation. This is clear from the fact that the reference 
shows that the cameras can be adjusted by loosening the screws fitted in the elongated 
apertures; see col.3, lines 50, et seq. 

It appears from page 3 of the Office Action that the Miura reference is understood to 
show both cameras can be adjusted in left-right direction. Specifically, the Office Action 
considers Miura as showing each camera can be rotatably adjusted with its optical axis as a 
center of rotation and that each camera can be adjusted in a left-right direction. 

Considering only the description in col. 3, lines 62-65, and col. 4, lines 7-9, of Miura 
that is referred to on page 3 of the November 18, 2004 Office Action, it could be interpreted 
that the optical axes of both cameras can be adjusted in left-right direction. However, a closer 
study of the reference in its entirety shows that each camera is rotatably adjusted with its 
optical axis as a center of rotation. 

As shown in the description in col. 3, line 51, to col. 4, line 9, of Miura, the arrows "E" 
or "F" show the adjustment direction in Fig. 4b. This shows an adjustment direction of camera 
20a by movement of screws 82, 84 within the elongated holes 86, 88, and also shows an 
adjustment direction of camera 20b by movement of screws 100, 102 within the arch shaped 
elongated hole (not shown). Thus, even if the cameras are adjusted in this "E" or "F" adjustment 
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direction, the optical axis of camera is always parallel with the shooting direction and not as 
defined in Claim 1 herein; namely, incUned toward the main camera side with a predetermined 
angle with respect to the shooting direction defined by each of the optical axes and the shooting 
direction. 

Please note that an adjustment of the optical axis in left-right direction in Miura is made 
in the "C" or "D" direction as shown in Fig. 3A. 

Miura does not disclose that both cameras can be adjusted in left-right direction, and does 
not suggest that optical axes of both cameras be inclined toward the main camera side with a 
predetermined inclination angle with respect to the shooting direction or vehicle central axis. 

The rejection of Claims 1, 3, 5, 9 and 16 under 35 U.S.C. § 103(a), in view of Miura, is 
traversed and reconsideration is respectftiUy requested. 

The Official Action admits that Miura does not show that the optical axes of the two 
cameras are inclined toward the main camera side at a predetermined angle defined by the 
optical axes and the shooting direction. 

Not only that, but the Miura reference also does not obtain a three-dimensional distance 
distribution by image processing (stereo matching) as defined in the present invention. In 
addition, Miura does not disclose searching the corresponding area in a striplike search area as 
defined in Claim 1. According to Miura, the operator is able to see the three-dimensional 
distance distribution by using the Hquid crystal shutter glasses (3-D glasses). 

However, to obtain the high-precision three-dimensional distance distribution by the 
image processing (stereo matching), the three-dimensional distance distribution can be obtained 
only in a part of the region imaged by both cameras. In this structure, it is difficult to specify the 
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region of the comparison image that coincides with the small region of the reference image, and 
there is no choice other than the process that the region with the highest coincident region among 
plural candidates is judged as the coincident region. 

More specifically, in the stereo matching, the correlation value is calculated for every 
shift while sequentially shifting the small region in the reference image with respect to the 
predetermined search area in the comparison image. Then, the calculated correlation values are 
compared with each other, and the shift amount which has the highest correlation value is 
defined. Therefore, it is important to obtain the predetermined length search area in the 
comparison image. 

There is no reason, motivation or suggestion for a person skilled in the art to change or 
modify the Miura system to include the features of the inclined optical axes, image processing 
means for calculating a three dimensional distance distribution and searching a striplike search 
area. Therefore, Miura clearly fails to estabhsh prima facie obviousness of the subject matter of 
Claim 1. 

Claims 3, 5, 9 and 16 depend fi-om Claim 1 and define fiirther features of the invention. 
In view of the fact that Miura fails to establish prima facie obviousness for the subject matter as 
explained above, applicants submit that, likewise, Miura fails to establish prima facie 
obviousness for the subject matter of Claims 3, 5, 9 and 16. 

With regard to dependent Claim 3, this claim has a feature such that the optical axis of 
the sub-camera is inclined toward the sub-camera side with respect to the optical axis of the main 
camera. In col. 1, lines 39-47, of Miura, each of the cameras needs to be adjusted about its 
optical axes so that left and right cameras images coincide with each other on an "imaginary 
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plane" in the predetermined distance. In contrast, the present invention performs the adjustment 
so that the left and right cameras images do not coincide v^ith each other on "imaginary plane" in 
the predetermined distance. Thus, Miura does not suggest the above-mentioned features of the 
present invention. 

Applicants further note that dependent Claims 3 and 9 have an optical axis of the sub- 
camera that is inclined toward the sub-camera side with respect to the main camera, which is not 
disclosed or suggested in any of cited references. 

Miura does not disclose a structure that both cameras are inclined to left or right together, 
but discloses that one of cameras is incUned to left or right so that the predetermined 
convergence angle is obtained. Moreover, Miura does not disclose or suggest the above- 
mentioned feature of "the optical axis of the sub-camera is inclined toward the sub-camera side 
with respect to the optical axis of the main camera" in Claim 3. 

Dependent Claim 16 specifies that the "angles of inclination of said main camera and said 
sub-camera are set to be such angles that make an area substantially left-right symmetric with 
respect to a central axis of a vehicle parallel to the shooting direction, said area being an area of 
three-dimensional distance distribution obtained by an image processing unit on the basis of 
images photographed by said cameras". 

The Office Action sets forth that Figure 6 of Miura discloses that if the shooting direction 
and the central axis of the vehicle coincide with each other, the three dimensional distance 
distribution becomes inherently left-right symmetric. 

Applicants, however, contend that Miura performs the detection in an area where the 
reference and comparison images coincide with each other based upon someone's judgment 
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using the liquid crystal shutter spectacles. In contrast, the present invention has a device that 
detects the corresponding area, i.e. an image processing device. 

The rejection of Claims 10 and 13 under 35 U.S.C. § 103(a) in view of Miura, taken with 
Tanigawa (US Patent 5,915,033), is traversed and reconsideration is respectfully requested. 

Miura is discussed above and all comments apply with equal emphasis to the rejection of 
Claims 10 and 13. 

Although the Tanigawa reference discloses that the three-dimensional distance 
distribution can be obtained by the image processing (stereo matching), Tanigawa" s system is 
different fi-om the image processing (stereo matching) recited in Claim 1, upon which Claims 10 
and 13 ultimately depend. 

Thus, the stereo matching of Tanigawa divides the reference image and the comparison 
image into a window frame WDl and the window fi-ame WD2, respectively. The window firame 
WDI of the reference image and the window frame WD2 of the comparison image have the same 
region. 

On the other hand, the present invention as now defined specifies that the search area set 
in the comparison image with respect to the small region of the reference image extends in a 
striplike shape from the position corresponding to the small region. 

Tanigawa does not disclose or suggest the structure where the search area set in the 
comparison image extends in a striplike shape fi-om the position corresponding to the small 
region. 

Thus, the cited combination of Miura and Tanigawa do not disclose the following 
features defined in the claims of the present application: 
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(a) optical axes of both cameras are inclined toward the main camera side with a 
predetermined inclination angle with respect to the shooting direction; 

(b) image processing means for calculating a three-dimensional distance distribution of 
the object based on a positional difference between a region in the reference image and a 
corresponding area in the comparative image; and 

(c) search area set in the comparison image extends in a striplike shape from the position 
corresponding to the small region. 

As noted above, the stereo matching of Tanigawa divides the entire area of the reference 
image with the window frame WDl having the same area as the window frame WOl of the 
comparison image, and calculates the parallax for every window frame WDL The three- 
dimensional distance distribution obtained in the reference image by using this method, can be 
obtained for the entire area of the reference image, and it does not displace toward the 
comparison image as shown in the present invention, hi other words, the three-dimensional 
distance distribution obtained without the inclination of the optical axis of the camera with 
respect to the shooting direction can be left-right symmetric with respect to the shooting 
direction. 

However, if the window frame WDl of the reference image is set to have the same area 
as the window frame WD2 of the comparison image as shown in Tanigawa, the number of the 
sensor data that the window frame WDl includes is increased. Since only one parallax is 
obtained for each window frame WDl in the stereo matching of Tanigawa, if the window 
frame WDl includes many sensor data, these sensor data are considered to have the same 
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parallax. Therefore, the parallax obtained for each window frame WDl is of low precision and, 
thus, the three-dimensional distance distribution is of low precision. 

On the other hand, in the present invention, the search area of the comparison image 
extends from the position corresponding to the small region, and optical axes of both cameras 
are inclined towards the main camera side with the predetermined inclination angle with respect 
to the shooting direction. Thus, the present invention has an effect that the high precision three- 
dimensional distance distribution can be obtained while reducing the number of sensor data 
which are considered to have the same parallax, and the three-dimensional distance distribution 
can be made left-right symmetric with respect to the shooting direction. 

There is no reason, suggestion or motivation in either Miura or Tanigawa whereby a 
person skilled in the art would be lead to change or modify the Miura system and to thereby 
arrive at the present invention. Therefore, no prima facie obviousness has been established. 

Dependent Claims 9, 10 and 13 

The common feature of dependent Claims 9, 10 and 13 is that the sub-camera is 
inclined toward the sub-camera side with respect to the optical axis of main camera. With this 
structure, an infinite distance corresponding point with respect to a small region located in an 
image end portion of the reference image can be obtained with certainty in the comparison 
image. 

Miura discloses that an optical axis of one of two cameras is adjusted so as to obtain a 
predetermined convergence angle defined by the intersection of the optical axes of the cameras. 
Therefore, the adjustment of the Miura cameras is not performed with the structure as defined by 
the present claims under any circumstances. 
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Claims 10 and 13 have a feature that a search margin is provided for searching an 
infinite distance corresponding point corresponding to a small region located near the end of the 
reference image. This feature is not disclosed in any cited reference. 

Miura shows a structure wherein the optical axis of one of two cameras is adjusted so as 
to obtain a predetermined convergence angle defined by the intersection of the opfical axes of the 
cameras. With such an adjusted angle, if there is a variation in imaging areas of the respective 
cameras due to manufacturing error, an infinite distance corresponding point corresponding to 
the small region located at the end of the reference image does not exist within the comparison 
image, and, thus, the parallax of the infinite distance corresponding point carmot be calculated. 

The correcfion of deviafion in the Tanigawa device is accomplished in that the 
calculated parallax is corrected based on the correction coefficient corresponding to the deviation 
between the position of a region in a reference image and the position of its corresponding region 
in the comparison image. 

With this structure of the Tanigawa reference, the correct parallax can be obtained by 
the above-mentioned correction it the coincident region can be defined correctly. However, even 
if this correction is made, if there is the variation in imaging areas by both cameras, the parallax 
of an infinite distance corresponding point corresponding to the small region located at the end 
of the reference image cannot accurately be calculated. In other words, if the deviation occurs in 
an optical axis direction of each camera due to an error in assembly, and thus, an infinite distance 
corresponding point corresponding the small region of the reference image does not exist within 
the comparison image, the parallax of the infinite distance corresponding point cannot be 
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calculated because the different region which is not the proper coincident region is defined 
mistakenly as the coincident region. 

On the other hand, since the present invention provides the search margin for searching 
an infinite distance corresponding point corresponding to the small region located at the end of 
reference image, the present invention has an effect that the infinite distance corresponding point 
corresponding to the small region located at the end of the reference image can accurately exist 
within the comparison image, and thus parallax can be accurately calculated. 

In Miura, even if the main camera captures the target object on the main camera side, 
outside of the viewing area of the sub-camera, the optical axes of both of cameras are not 
inclined towards the main camera side as required in the present invention. In the present 
invention, the effect obtained by the above-mentioned feature, as described in page 1 1, lines 4-9, 
of applicants' specification, is obtaining the three-dimensional distance distribution having left- 
right symmetry with respect to the shooting direction can be obtained. Miura, however, does 
not disclose the above-mentioned features and the resulting benefits obtained thereby. 

Applicants contend that Miura has a structure having an optical axis of the sub-camera 
that is inclined towards the main camera side with respect to the optical axis of the main camera, 
contrary to the present invention, and that the infinite distance corresponding point which 
corresponds to the small area at edge of the reference image does not exist within the comparison 
image. 

According to applicants, the secondary reference, Tanigawa, discloses that by assembling 
errors of each of the cameras, a deviation occurs between the position of the area in the reference 
image and the position of the corresponding area in the comparison image. With the correction 
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coefficient previously specified by taking the deviation into the consideration, a correction is 
performed for the calculated parallax so that the parallax can accurately calculated. 

In the Tanigawa device, after searching the area where the image data coincide w^ith each 
other, the correction is performed in the calculation of the parallax, and thus, when the area 
where the image data coincide with each other can not be searched, and the parallax cannot be 
correctly calculated even if the correction is performed. That is, if the indefinite distance 
corresponding point which corresponds to the small area at the edge of the reference image does 
not exist within the comparison image due to the assembling errors of the stereo camera, the 
parallax of the infinite distance corresponding point can not be calculated. 

hi contrast, the present invention has a structure that by inclining the optical axis of sub- 
camera towards the sub-camera side with respect to the optical axis of the main camera, the 
infinite distance corresponding point which corresponds to the small area at edge portion of the 
reference image can surely exist within the comparison image, and thus, the infinite distance 
corresponding point can be detected even if the there are the assembling errors. 

Further, applicants note that in Claims 10 and 13, the indefinite distance corresponding 
point which corresponds to the small area at the edge of the reference image can be detected. 
These claims have a feature such that in order to detect the infinite distance corresponding point, 
the optical axis is inclined toward the sub-camera side with respect to the optical axis of the main 
camera. 

Further, based on the description in page 8, lines 8-21 and Figure 4, new dependent 
Claim 21 defines that the search area is longer than the length of the small region of the reference 
image. 
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The inclination amount of both cameras and the structure that the three-dimensional 
distance distribution becomes left-right symmetric with respect to the shooting direction by 
including the camera which is now defined in new Claim 22 is not rendered prima facie obvious 
by the combination of references. 

More specifically, based on the descripfion from page 9, line 21 to page 10, line 8 and 
Figure 2D, Claim 22 recites the inclinafion angle of both cameras is an angle corresponding to a 
displacement between a line for setting the three-dimensional distance distribution generating 
area on the reference image substantially symmetrical on the left and right sides and a vertical 
line perpendicular to the optical axis of the main camera on the reference image. This feature is 
neither shown nor suggested by the cited prior art. 

With regard to the additional citations of Lipton (US 6,063,441) and Saneyoshi (US 
5,410,346) and the rejecdons of Claims 6 and 7, these claims depend on Claim 1 and the 
discussion of Miura above applies with equal emphasis here. 

The Lipton and Saneyoshi references would not lead persons skilled in the art to make 
changes or modifications to the Miura system with the expectation of a desirable result that 
would fall within the scope of the rejected claims. 
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Applicants respectfully submit that this amendment and the above remarks obviate the 
outstanding objections and rejections in this case, thereby placing the application in condition for 
immediate allowance. Allowance of this application is earnestly solicited. 



Suite 3100, Promenade II 
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CONCLUSION 



Respectfully submitted, 



SMITH, GAMBRELL & RUSSELL, LLP 
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